In a computer system, memory storage accessible to a processor may be implemented as a memory hierarchy, which includes multiple levels of memory devices having different storage sizes and access speeds. To reduce processing time, a faster memory device is placed at a higher level of memory hierarchy than a slower memory device. A memory device having a faster access speed tends to be more expensive per bit than a memory device having a slower access speed. Thus, the storage size of the faster memory device, which is placed at a higher level of memory hierarchy, may be smaller than the storage size of the slower memory device, which is placed at a lower level of memory hierarchy.
For example, in a computer system, a top or a first level of the memory hierarchy may include Static Random Access Memory (SRAM). This level of memory is typically referred to as a cache. A second level of the memory hierarchy includes a memory device that is slower but larger in storage size than SRAM, such as a Dynamic Random Access Memory (DRAM). This level of memory is typically referred to as a main memory. Additionally, a third level of the memory hierarchy includes an even slower memory device, such as a magnetic disc drive or a solid-state drive, having the largest storage size in the computer system. This level of memory hierarchy is typically referred to as a secondary storage.
To take advantage of the speeds and storage sizes in different levels of the memory hierarchy, information (such as data or programs) is divided into many segments and also stored in a hierarchical structure. For example, an entire executable program is stored at a lower level of the memory hierarchy. However, in general, some segments of the executable program are also stored at a higher level of the memory hierarchy. For example, those segments that are recently or frequently accessed are stored at a higher level of the memory hierarchy, which has higher access speed, to reduce access time.
To execute a certain function of a program, for example, a processor first requests from a first (or highest) level of the memory hierarchy a segment of the program that includes the function to be executed. If the requested segment exists at the first level, the processor executes the function included in the requested segment. This situation is called a “hit.” If the requested segment does not exist at the first level, the requested segment is copied from a lower level of memory hierarchy to the first level to be executed by the processor. This situation is called a “miss.”
Similarly, to access certain data, for example, a processor first requests from a first (or highest) level of the memory hierarchy a segment of the data to be accessed. If the requested segment exists at the first level, the processor accesses the requested segment. This situation is called a “hit.” If the requested segment does not exist at the first level, the requested segment is copied from a lower level of memory hierarchy to the first level to be accessed by the processor. This situation is called a “miss.”
The above described memory management scheme can be implemented between a cache memory and a main memory, and the scheme is known as cache. A similar scheme, known as virtual memory, can be implemented to manage memory resources between a main memory and a secondary storage.